大量证据表明，深神经网络（DNN）容易受到后门攻击的影响，这激发了后门检测方法的发展。现有的后门检测方法通常是针对具有单个特定类型（例如基于补丁或基于扰动）的后门攻击而定制的。但是，在实践中，对手可能会产生多种类型的后门攻击，这挑战了当前的检测策略。基于以下事实：对抗性扰动与触发模式高度相关，本文提出了自适应扰动生成（APG）框架，以通过自适应注射对抗性扰动来检测多种类型的后门攻击。由于不同的触发模式在相同的对抗扰动下显示出高度多样的行为，因此我们首先设计了全球到本地策略，以通过调整攻击的区域和预算来适应多种类型的后门触发器。为了进一步提高扰动注入的效率，我们引入了梯度引导的掩模生成策略，以寻找最佳区域以进行对抗攻击。在多个数据集（CIFAR-10，GTSRB，Tiny-Imagenet）上进行的广泛实验表明，我们的方法以大幅度优于最先进的基线（+12％）。

在过去的几年中，从面部视频中对心脏脉搏的测量已成为对研究的有趣追求。这主要是由于以非侵入性方式获得个人心率的重要性越来越重要，这对于游戏和医疗行业的应用可能非常有用。在过去的几年中，研究的另一个工具领域是深度学习的出现，并使用深度神经网络来增强任务绩效。在这项工作中，我们建议使用有效的卷积网络来准确测量低分辨率面部视频的用户心率。此外，为了确保我们能够实时获得心律，我们通过修剪深度学习模型来压缩深度学习模型，从而减少其内存足迹。我们在MAHNOB数据集上基准了方法的性能，并在多种方法中比较了其性能。

面部视频中心率的估计在医疗和健身行业中有许多应用。此外，它在游戏领域也变得有用。已经提出了几种方法，可以从面部视频中无缝获得心率，但是这些方法在处理运动和照明工件方面存在问题。在这项工作中，我们使用用户的光谱反射率提出了一个可靠的人力资源估计框架，这使运动和照明干扰变得强大。我们采用基于学习的深度框架，例如更快的RCNNS来执行面部检测，而不是先前方法使用的中提琴琼斯算法。我们在Mahnob HCI数据集上评估了我们的方法，发现所提出的方法能够超越先前的方法。从面部视频中估计心率在医疗和健身行业中有许多应用。此外，它在游戏领域也变得有用。已经提出了几种方法，可以从面部视频中无缝获得心率，但是这些方法在处理运动和照明工件方面存在问题。在这项工作中，我们使用用户的光谱反射率提出了一个可靠的人力资源估计框架，这使运动和照明干扰变得强大。我们采用基于学习的深度框架，例如更快的RCNNS来执行面部检测，而不是先前方法使用的中提琴算法。我们在MAHNOB HCI数据集上评估了我们的方法，发现所提出的方法能够超过以前的方法。

自动摘要方法是有效的，但可能患有低质量。相比之下，手动摘要很昂贵，但质量更高。人类和人工智能可以协作以提高总结性能吗？在类似的文本生成任务（例如机器翻译）中，人类AI合作的形式是“后编辑” AI生成的文本，可减少人类的工作量并提高AI输出的质量。因此，我们探讨了邮政编辑是否提供文本摘要中的优势。具体来说，我们对72名参与者进行了实验，将提供的后编辑摘要与手动摘要进行了摘要，以摘要质量，人为效率和用户在正式新闻（XSUM新闻）和非正式（REDDIT帖子）文本方面进行了比较。这项研究对何时编辑的文本摘要提供了宝贵的见解：在某些情况下（例如，何时参与者缺乏领域知识），但在其他情况下却没有帮助（例如，何时提供的摘要包括不准确的信息）。参与者的不同编辑策略和援助需求为未来的人类摘要系统提供了影响。

Domain adaptation enables the learner to safely generalize into novel environments by mitigating domain shifts across distributions. Previous works may not effectively uncover the underlying reasons that would lead to the drastic model degradation on the target task. In this paper, we empirically reveal that the erratic discrimination of the target domain mainly stems from its much smaller feature norms with respect to that of the source domain. To this end, we propose a novel parameter-free Adaptive Feature Norm approach. We demonstrate that progressively adapting the feature norms of the two domains to a large range of values can result in significant transfer gains, implying that those task-specific features with larger norms are more transferable. Our method successfully unifies the computation of both standard and partial domain adaptation with more robustness against the negative transfer issue. Without bells and whistles but a few lines of code, our method substantially lifts the performance on the target task and exceeds state-of-the-arts by a large margin (11.5% on Office-Home [45] and 17.1% on VisDA2017 [31]). We hope our simple yet effective approach will shed some light on the future research of transfer learning. Code is available at https://github.com/jihanyang/AFN .

A recent study has shown a phenomenon called neural collapse in that the within-class means of features and the classifier weight vectors converge to the vertices of a simplex equiangular tight frame at the terminal phase of training for classification. In this paper, we explore the corresponding structures of the last-layer feature centers and classifiers in semantic segmentation. Based on our empirical and theoretical analysis, we point out that semantic segmentation naturally brings contextual correlation and imbalanced distribution among classes, which breaks the equiangular and maximally separated structure of neural collapse for both feature centers and classifiers. However, such a symmetric structure is beneficial to discrimination for the minor classes. To preserve these advantages, we introduce a regularizer on feature centers to encourage the network to learn features closer to the appealing structure in imbalanced semantic segmentation. Experimental results show that our method can bring significant improvements on both 2D and 3D semantic segmentation benchmarks. Moreover, our method ranks 1st and sets a new record (+6.8% mIoU) on the ScanNet200 test leaderboard. Code will be available at https://github.com/dvlab-research/Imbalanced-Learning.

Weakly-supervised object localization aims to indicate the category as well as the scope of an object in an image given only the image-level labels. Most of the existing works are based on Class Activation Mapping (CAM) and endeavor to enlarge the discriminative area inside the activation map to perceive the whole object, yet ignore the co-occurrence confounder of the object and context (e.g., fish and water), which makes the model inspection hard to distinguish object boundaries. Besides, the use of CAM also brings a dilemma problem that the classification and localization always suffer from a performance gap and can not reach their highest accuracy simultaneously. In this paper, we propose a casual knowledge distillation method, dubbed KD-CI-CAM, to address these two under-explored issues in one go. More specifically, we tackle the co-occurrence context confounder problem via causal intervention (CI), which explores the causalities among image features, contexts, and categories to eliminate the biased object-context entanglement in the class activation maps. Based on the de-biased object feature, we additionally propose a multi-teacher causal distillation framework to balance the absorption of classification knowledge and localization knowledge during model training. Extensive experiments on several benchmarks demonstrate the effectiveness of KD-CI-CAM in learning clear object boundaries from confounding contexts and addressing the dilemma problem between classification and localization performance.

Witnessing the impressive achievements of pre-training techniques on large-scale data in the field of computer vision and natural language processing, we wonder whether this idea could be adapted in a grab-and-go spirit, and mitigate the sample inefficiency problem for visuomotor driving. Given the highly dynamic and variant nature of the input, the visuomotor driving task inherently lacks view and translation invariance, and the visual input contains massive irrelevant information for decision making, resulting in predominant pre-training approaches from general vision less suitable for the autonomous driving task. To this end, we propose PPGeo (Policy Pre-training via Geometric modeling), an intuitive and straightforward fully self-supervised framework curated for the policy pretraining in visuomotor driving. We aim at learning policy representations as a powerful abstraction by modeling 3D geometric scenes on large-scale unlabeled and uncalibrated YouTube driving videos. The proposed PPGeo is performed in two stages to support effective self-supervised training. In the first stage, the geometric modeling framework generates pose and depth predictions simultaneously, with two consecutive frames as input. In the second stage, the visual encoder learns driving policy representation by predicting the future ego-motion and optimizing with the photometric error based on current visual observation only. As such, the pre-trained visual encoder is equipped with rich driving policy related representations and thereby competent for multiple visuomotor driving tasks. Extensive experiments covering a wide span of challenging scenarios have demonstrated the superiority of our proposed approach, where improvements range from 2% to even over 100% with very limited data. Code and models will be available at https://github.com/OpenDriveLab/PPGeo.

In this work, we focus on instance-level open vocabulary segmentation, intending to expand a segmenter for instance-wise novel categories without mask annotations. We investigate a simple yet effective framework with the help of image captions, focusing on exploiting thousands of object nouns in captions to discover instances of novel classes. Rather than adopting pretrained caption models or using massive caption datasets with complex pipelines, we propose an end-to-end solution from two aspects: caption grounding and caption generation. In particular, we devise a joint Caption Grounding and Generation (CGG) framework based on a Mask Transformer baseline. The framework has a novel grounding loss that performs explicit and implicit multi-modal feature alignments. We further design a lightweight caption generation head to allow for additional caption supervision. We find that grounding and generation complement each other, significantly enhancing the segmentation performance for novel categories. We conduct extensive experiments on the COCO dataset with two settings: Open Vocabulary Instance Segmentation (OVIS) and Open Set Panoptic Segmentation (OSPS). The results demonstrate the superiority of our CGG framework over previous OVIS methods, achieving a large improvement of 6.8% mAP on novel classes without extra caption data. Our method also achieves over 15% PQ improvements for novel classes on the OSPS benchmark under various settings.

Nearest-Neighbor (NN) classification has been proven as a simple and effective approach for few-shot learning. The query data can be classified efficiently by finding the nearest support class based on features extracted by pretrained deep models. However, NN-based methods are sensitive to the data distribution and may produce false prediction if the samples in the support set happen to lie around the distribution boundary of different classes. To solve this issue, we present P3DC-Shot, an improved nearest-neighbor based few-shot classification method empowered by prior-driven data calibration. Inspired by the distribution calibration technique which utilizes the distribution or statistics of the base classes to calibrate the data for few-shot tasks, we propose a novel discrete data calibration operation which is more suitable for NN-based few-shot classification. Specifically, we treat the prototypes representing each base class as priors and calibrate each support data based on its similarity to different base prototypes. Then, we perform NN classification using these discretely calibrated support data. Results from extensive experiments on various datasets show our efficient non-learning based method can outperform or at least comparable to SOTA methods which need additional learning steps.